Surprise-modulated belief update: how to learn within changing environments?
نویسندگان
چکیده
Abstract I We propose a new framework for surprise-driven learning that can be used for modeling how humans and animals learn in changing environments. It approximates optimal Bayesian learner, but with significantly reduced computational complexity. I This framework consists of two components: (i) a confidence-adjusted surprise measure to capture environmental statistics as well as subjective beliefs, (ii) a surprise-minimization learning rule, or SMiLe-rule, which dynamically adjusts the balance between new and old information for belief update.
منابع مشابه
A Novel Measure of Surprise with Applications for Learning within Changing Environments
Surprise is informative because it drives attention [1] and modifies learning [2]. Correlates of surprise have been observed at different stages of neural processing, and found to be relevant for learning and memory formation [3]. Although surprise is ubiquitous, there is neither a widely accepted theory that quantitatively links surprise to observed behavior, such as the startle response, nor ...
متن کاملA novel information theoretic measure of surprise
Surprise is informative because it drives attention [IB09] and modifies learning [SD00]. Correlates of surprise have been observed at different stages of neural processing, and found to be relevant for learning and memory formation [RR03]. Although surprise is ubiquitous, there is neither a widely accepted theory that quantitatively links surprise to observed behavior, such as the startle respo...
متن کاملThe Benefits of Surprise in Dynamic Environments: From Theory to Practice
Artificial agents engaged in real world applications require accurate resource allocation strategies. For instance, open systems may require artificial agents with the capability to filter out all information which are irrelevant with respect to the actual intentions and goals. In this work we develop a model of surprise-driven belief update. We formally define a strategy for epistemic reasonin...
متن کاملNeural Mechanisms of Updating under Reducible and Irreducible Uncertainty.
Adaptive decision making depends on an agent's ability to use environmental signals to reduce uncertainty. However, because of multiple types of uncertainty, agents must take into account not only the extent to which signals violate prior expectations but also whether uncertainty can be reduced in the first place. Here we studied how human brains of both sexes respond to signals under condition...
متن کاملThe Effect of Syntactic Form on Simple Belief Revisions and Updates
In this paper we report preliminary results on how people revise or update a previously held set of beliefs. When intelligent agents learn new things which conflict with their current belief set, they must revise their belief set. When the new information does not conflict, they merely must update their belief set. Various AI theories have been proposed to achieve these processes. There are two...
متن کامل